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INTRODUCTION 



In the ten years that NCSALL was in existence, those of us in charge of disseminating 
research discovered that one of the most misunderstood terms in our field was the term 
evidence-based practice. With the advent of the No Child Left Behind (NCLB) 
legislation, there has been intensive pressure on those in education, including the adult 
education and literacy community, to base instruction and services on evidence-based 
practices. Simultaneously, the U.S. Department of Education has strongly (some would 
say almost exclusively) promoted the idea of scientifically based research, which was 
defined under the NCLB legislation. 

One would be hard-pressed now to find adult basic education practitioners or 
programs that don’t know they are supposed to be basing their practices on evidence. At 
the same time, few can accurately define the term evidence-based practice, let alone 
understand how one employs such practices in classrooms and programs. The 
nervousness among adult educators, who somehow felt that every decision made in their 
lessons must be backed up by a randomized-controlled trial, has been a barrier to the very 
adoption of such practices in our field. 

To address this situation, in 2006, the Institute of Education Sciences and the 
National Institute for Literacy funded NCSALL to develop a one-day workshop for adult 
basic education, TANL (Transitional Assistance to Needy Lamilies), and One-Stop 
practitioners and administrators. The goal of this workshop was to help teachers and 
administrators understand the meaning of evidence-based practice and develop strategies 
for continuously accessing, understanding, judging and using research in their classrooms 
and programs. 

The developers and presenters of the bCz-hour workshop were Beth Bingman 
(Universtiy of Tennessee, The Center for Literacy Studies/NCSALL), Cristine Smith 
(World Education/NCSALL), Lennox McLendon (National Adult Education 
Provessional Development Consortium), and John Comings (Harvard Graduate School of 
Education/NCSALL) . 

In each of three pilot states — North Carolina (July 18, 2006), Texas 
(September 12, 2006), and Wyoming (October 19, 2006) — NCSALL /NAEPDC 
conducted a workshop with approximately 25 practitioners and program 
administrators (along with selected professional developers and state staff 
participating as observers) that was intended to enable them to: 

• Discuss their own attitudes about research and its connection to practice 
and policy 

• Define some basic concepts and terminology about research design 
and methodology 
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. Describe the connection between evidence-based practice and 
program/classroom improvement 

• Describe what empirical evidence looks like and how it can be integrated 
with professional wisdom to make decisions about instruction and services 
for adult students 

• Implement a plan for improving reading instruction in their programs utilizing 
evidence-based practices 

. Cite strategies they can use to continuously access, understand, judge and use 
research to make decisions about practice in their classrooms or programs 

The workshop guide is available to the field and can be found at 
www.ncsall.net/fileadmin/resources/teach/EBP_wkshp_guide.pdf, and the PowerPoint 
slides that accompany it can be found at 

www.ncsall.net/fileadmin/resources/teach/EBP_wkshp_PPT.ppt. A one-page summary 
of the workshop agenda can be found in Appendix A of this report. 

As part of piloting this workshop, we conducted an evaluation of the participants’ 
experience, the knowledge they gained, to both determine how to revise the workshop 
and to gauge its effectiveness. This purpose of this paper is to report on the findings from 
that evaluation. 
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We constructed a simple pre- and post- workshop, one-page questionnaire, aligned with 
the workshop objectives presented in the previous section (pp. 1-2). The pre- and post- 
workshop questionnaires can be found in Appendix B of this report. Each included eight 
knowledge and practice questions about evidence-based practice, scientifically based 
research, types of research studies, research terms, as well as where and how often 
practitioners sought out research. The post- workshop questionnaire also included a 
question about the key concept practitioners would take away from the workshop. 

We gave out the pre- workshop questionnaire to participants as they entered the 
workshop and asked them to fill it out. At the end of the workshop, we handed out the 
post- workshop questionnaire and asked participants to fill that out. In both cases, the 
questionnaires took between 10 and 15 minutes for participants to complete. We ended 
up with 68 usable questionnaires. (See Appendix C for results.) 

We did not collect any demographic or background data from participants, 
since our goal with this evaluation was just to gauge what participants learned during 
the workshop. 
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• Definition of Evidence-based Practice : Overall, participants gained a better 

understanding of the term evidence-based practice. Most participants understood that 
it had something to do with basing their instruction on research, but few started the 
workshop understanding that it is “the integration of the best empirical evidence with 
professional wisdom in making decisions about instruction”, which is the definition 
put forward by the U.S. Department of Education (see workshop powerpoint slides 
for source). However, after the workshop, more participants understood that the basis 
of practice was not only research findings but also professional wisdom gained by the 
field. Examples of before and after follow: 



Pre-workshop: Define evidence- 
based PRACTICE 


Post-workshop: Define evidence- 
based PRACTICE 


Instructional practices that have been proven 
effective by research 


Integration of professional wisdom and 
scientifically based research to come up with 
best practices 


Practice based on prior past experience of 
what works and what doesn’t 


Practice based on available research and 
professional wisdom 


Practices that have been researched and 
have evidence 


Integration of professional wisdom with the 
best empirical evidence 


Using ideas/methods that have been 
tried by others that have given results 
from their practices 


Employing professional wisdom with evidence 


Research based on actual practice 


Implementing practices which are based on 
both research and teacher knowledge 



. Definition of Scientifically Based Research : Very few participants could cite the 
criteria for scientifically based research before the workshop started; scientifically 
based research, as defined by No Child Eeft Behind legislation, must be published in 
a peer-reviewed journal or approved by a panel of independent experts; the results of 
the study must have been replicated by other scientists; or there is consensus in the 
research community that the study's findings are supported by a critical mass of 
additional studies. This concept was harder to grasp for participants than the term 
evidence-based practice; still, it is clear that at least some participants began to 
understand the difference between research and scientifically based research. 



Pre-workshop: Criteria for scientifically 

BASED RESEARCH 


Post-workshop: Criteria for scientifically 

BASED RESEARCH 


Validity 


Validity 


Reliability 


Reliablility 
Random sample 
Peer reviewed 


Control group 


Peer reviewed 


Study group 


Replicated 


Meta-analysis. Case studies have 
been performed 


Published? Replicated? 
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Pre-workshop: Criteria for scientifically 

BASED RESEARCH 


Post-workshop: Criteria for scientifically 

BASED RESEARCH 


Been implemented in several settings. 
Placebos have been used so that research 
wasn’t manipulated. 


Been researched in more than one research. 
Published. 


A control group. A need for the type of 
research done. 


Published in an accredited journal. 
Have results of study been replicated 
by other scientists. 



. Differences in Research Methodologies : Participants’ concepts about different 
types of research methodology — case studies, correlational studies, and true 
experiments — were somewhat “muddy” coming into the workshop. Most 
participants chose to define what a case study was in the pre-workshop 
questionnaire, but those definitions were confused with qualitative analysis in 
general. While only a handful tried (and succeeded) in defining what true 
experiments are at the start of the workshop, more attempted and succeeded at the 
end. However, correlational studies were still difficult for participants to describe 
and define, indicating that the workshop was not as successful at differentiating that 
type of methodology from case studies and true experiments. 



Pre-workshop: Define case studies, 

CORRELATIONAL STUDIES OR TRUE EXPERIMENTS 


Post-workshop: Define case studies, 

CORRELATIONAL STUDIES OR TRUE EXPERIMENTS 


Case studies — specific study with hypothesis, 
procedures and outcomes. 


Case studies: detailed description of individual 
or small groups: “Whafs going on?” 


Case studies: looking at groups in their natural 
settings to do the research. 


True experiments: randomized tests that 
answer what works. 


Case studies: people. Information on 
individuals recorded and studied over a 
period of time. Correlational: comparison of 
case studies. 


Case studies: individual/group descriptions. 
Correlational: tests links between variables 
and outcomes. True experiment: randomized 
trials that answer “what works” questions. 


Correlational studies: studies conducted to 
prove a theory. 


True experiments: done with a control group 
and an intervention group. 


In case studies, history and current 
practice are included and results are 
compared and controlled to other methods 
with comparable histories. 


True experiments: Control group, participant 
group — measure results of methods used. 
Random selection of both groups. 



. Definition of Experimental and Control : About half of the participants came in 
understanding the difference between a control group and an experimental group. 
The workshop appears to have been successful both at clarifying some 
misconceptions about what a control group is and at helping more participants 
understand the basics of how a sample in research is constructed. 



Pre-workshop: Difference between 

CONTROL AND EXPERIMENTAL GROUP 


Post-workshop: Difference between 

CONTROL AND EXPERIMENTAL GROUP 


Experimental: behaviors of study group 
are unbiasedly observed. Control: defined 
and managed study population with 
predicted outcomes. 


Experimental: population exposed to 
enhancement variable. Control: population not 
exposed to “enhancement variable.” 
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Pre-workshop: Difference between 

CONTROL AND EXPERIMENTAL GROUP 


Post-workshop: Difference between 

CONTROL AND EXPERIMENTAL GROUP 


Control group is monitored more. More rules 
about who can make up the group. 


Control group remains same and 
experimentai: group received additional 
aids/training to see if there’s a difference. 


(Left blank) 


The experimental group receives the 
intervention strategies. 


Experimental: an open-ended study. A control 
group is followed ail the way through a study. 


Control continues same mode. Experimental 
has intervention. 


Experimental: you experiment with unknown 
variables. Control: experiment with known 
variables (you control the variables) 


Experimental: group has interventions and 
control group does not. 



. The Importance of Random Assignment : The majority of participants, at the start of 
the workshop, felt that random assignment was important in selecting a sample 
because it somehow “reduces bias”, but could not say more than this. At the end of 
the workshop, there was some evidence that more participants understood the true 
purpose of random selection, although many still confusd the terms sample and 
population and many still couldn’t completely state the rationale clearly. 



Pre-workshop: Importance of random 

ASSIGNMENT IN CHOOSING STUDY SAMPLE 


Post-workshop: Importance of random 

ASSIGNMENT IN CHOOSING STUDY SAMPLE 


Research validity is based on a comparison to 
underlying variation in the population. 


So that each group is equally representative 
of the total population. 


Because it allows for a truer “picture.” 


Because it provides for a wider 
distribution/comparison to the 
actual population. 


This gives you a true outcome of how a new 
project would affect most other programs. 


You get a true group that represents a 
general population. 


It depends on whafs being researched. 


Self-selected participants may have a bias 
or pre-determined ideas. 



. How to Access Research and Extent of Research Use in Practice : The trend was for 
most participants to indicate that they would use research more frequently at the end 
of the workshop than at the beginning. One change the workshop clearly engendered 
was a broader understanding of where to find research. The handouts in the workshop 
about various websites and sources of research information appeared to be helpful to 
participants. 



• Key Questions to Ask about Particular Studies : A good number of participants, pre- 
workshop, indicated that their main question was “Is this research relevant to the 
adult students with whom I work?” At the end of the workshop, some participants 
still cited this as a key question, which is legitimate, but a few others added questions 
they would ask about the validity and reliability of the study, about whether it had 
been published in a peer-reviewed journal, or whether it had been replicated. 
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Pre-workshop: Questions to ask about 

RESEARCH STUDIES 


Post-workshop: Questions to ask about 

RESEARCH STUDIES 


Who conducted, what controls. 


Was It published? Was It replicated? 
Supported by other research 
(significant amount)? 


Is It applicable to my students? Is It practical 
(In terms of cost, time, outcomes)? 


Are the results valid, reliable and practical? 


Is It applicable to our situation? 


Accepted by colleagues (published). 


Methodology. Relevance of research question. 


Methodology. Support by other studies. 
Quality of data. Peer reviewed. 



. Key Concept or Strategy Gained from the Workshop : This was the only question 
asked in the post- workshop questionnaire that was not asked in the pre- workshop 
questionnaire. The answers indicate a greater understanding of research, as well as a 
clearer knowledge of how to use research. 

What key concept or strategy do you walk away with from today’s training? 

• The importance of research and its validity. 

• Much better understanding of scientifically based research principles 
and application. 

• Be able to back up your professional wisdom with scientific research because 
it becomes more believable to others. 

• Importance of knowing where to find research and how to use it to 
improve instruction. 

• It’s not as difficult to incorporate research into our planning to help us meet 
our needs. 

• Using reliable research as basis for implementing change. 

• Scientific research can improve programs and teaching. 

• The need to keep sending research articles to staff and start 
discussion groups. 

• Research is not the end all. Instructor experience counts! Yeah. 

• Improved knowledge in analyzing data and research. 

• We ’re doing things right, research is easily accessible on the web. 

• Ways to implement better reading strategies. 

• Mean, median, and mode! Using research better to help me improve 
program delivery. 

• That research can make a difference in instructional practices. 

. There is a lot of research out there that can help us do a better job, if we will 
use it. 

• Accessing, understanding, judging and using research. 
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Align teaching with research. 

Alignment between research, instruction and assessment. 

Importance of moving from “by the seat of your pants” instruction. 
You can use research to make change. 




CXDNCLUSION 



From the findings reported here as well as from our experiences and interactions with the 
participants and state staff members in the pilot states, we believe that the Evidence- 
based Practice Workshop was effective in reaching its objective to enable participants to: 

• Discuss their own attitudes about research and its connection to practice 
and policy 

• Define some basic concepts and terminology about research design and 
methodology 

. Describe the connection between evidence-based practice and 
program/classroom improvement 

• Describe what empirical evidence looks like and how it can be integrated 
with professional wisdom to make decisions about instruction and services 
for adult students 

• Implement a plan for improving reading instruction in their programs utilizing 
evidence-based practices 

. Cite strategies they can use to continuously access, understand, judge and use 
research to make decisions about practice in their classrooms or programs 

Participants tended to leave the workshops sharing enthusiasm about implementing what 
they had learned about research utilization in their own programs. 

However, we would be remiss if we did not point out the limits of this 
evaluation and the limits of the workshop. The findings reported here are based on a 
one-time, pre-post questionnaire administered on the day of the workshop. The 
questionnaire went well beyond asking for a response to the workshop by querying 
content knowledge, but we can only report on changes in content knowledge. We 
cannot really say anything about actual changes in practices made by participants in 
these workshops. A more extensive effort that examines program changes over time 
would be needed to assess implementation. 

The workshop was designed to be completed in a one-day session. This design 
has its own limits. Professional development efforts that extend over time and have 
program change efforts built into and supported in the design are more likely to lead to 
change in practices. Each of the three pilot states had their own plans to follow up with 
the participants in the pilot workshops and to support them to some extent. Tracking the 
results of these efforts and taking the results into account in any redesign of this 
workshop could strengthen the results. 
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1. Introductory Activity: Welcome, Objectives, and Agenda (15 minutes) 

A whole group activity where participants are introduced to each other, share a 
strength of their programs, and learn about the workshop as a whole. 

2. Attitudes Toward Research: Discussion Activity (35 minutes) 

A whole group discussion activity to help participants understand that there are a 
range of existing attitudes towards research and its role in improving practice. 
Participants analyze their own attitudes towards research and the attitudes they 
think other practitioners might hold, and they think about what attitudes about 
research mean for promoting evidence-based practice. 

3. Introduction to Research Design (70 minutes) 

An overview of the basics of research in experimental design and quantitative 
data analysis. Participants create their own data, explore how to analyze such data 
and learn basic research concepts such as random sampling, reliability, validity, 
means/medians/modes, standard deviation, findings and implications. The activity 
concludes with a group discussion of experimental, correlational, and case study 
research and when each might be useful to answer questions in adult education. 

4. Understanding Research and Identifying Evidence-based Practice 

(110 minutes) 

An activity where the concepts of scientifically based research and evidence- 
based practice are defined. Participants analyze research related to adult reading 
instruction, and generate principles of evidence-based reading instruction. 

5. Using Evidence-based Practice to Improve Your Program (45 minutes) 

A small group, program- specific activity where participants think about a recent 
change they made in their own program (and the steps involved), read a case 
study about a fictional program making a program improvement change based on 
an evidence-based practice about reading, and then compare their own change 
process with the case study to see how they might incorporate evidence-based 
practice in their program. 

6. Next Steps: Action Planning (40 minutes) 

An action planning activity in which participants talk about specific problems 
that they are currently facing in their programs for which reading research might 
be useful, how they will take the information on evidence-based reading back to 
their programs. 
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7. Accessing, Understanding, Judging and Using Research On Your Own 

(60 minutes) 

A whole group discussion activity where participants consider a strategy for how 
they will continuously access, understand, judge and use research as part of their 
ongoing efforts to improve their programs. 

8. Evaluation and Closure (10 minutes) 

An activity to get feedback from the participants about the workshop. 
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PRE-V\a^KSHOP QUEsiiasm 



First Name and Last Initial 

1 . How would you define evidence-based practice! 



2. Name two features or criteria of scientifically based research. 



3. Define one of these three types of research design: 

• case studies 

• correlational studies 

• true experiments 



4. What’s the difference between an experimental and a control group? 



5. Why do you think random assignment is so important in selecting a research sample? 

6. To what extent would you say you now use evidence from empirical research to 
inform your teaching and/or administration work in adult education? 

Never Daily 

1 2 3 4 5 6 

7. How do you access research now? 



8. What questions about a particular study would you ask to determine whether you 
should make a change in your practice based on the research findings? 
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First Name and Last Initial 

1 . How would you define evidence-based practice! 



2. Name two features or criteria of scientifically based research. 



3. Define one of these three types of research design: 

• case studies 

• correlational studies 

• true experiments 



4. What’s the difference between an experimental and a control group? 



5. Why do you think random assignment is so important in selecting a research sample? 



6. To what extent would you like to use evidence from empirical research to inform your 
teaching and/or administration work in adult education? 

Never Daily 

1 2 3 4 5 6 

7. How will you access research now? 



8. What questions would you ask about a particular study to determine whether you 
should make a change in your practice based on the research findings? 



9. What key concept or strategy do you walk away with from today’s training? 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 


1 


Define ebp 


Practical “work” that derives structure and method 
from observation and assessment to define best 
method 


Combo of professional wisdom 




2 features sbr 


Controlled variables 
Observation of behavior 


Data collection 




Define 1 type research 
des 


Case studies: comparison and assessment of 
existing data 


Case studies: written report on research for 
situational comparisons 




Diff exp & control group 


Exp: behaviors of study group are unbiasedly 
observed 

Control: defined and managed study population 
with predicted outcomes 


Exp: population exposed to enhancement variable 
Control: population not exposed to “enhancement 
variable” 




Random assign import? 


About 10% of total 


Allows unforeseen variables to be better digested 
into research results 




Now use/like to use 
eviden 


4 


5 




How access research 


From books 






?s ask about partic study 


Does “it” work in my situation? 


What are the inferences 




Key concept take away 




All great 


2 


Define ebp 


That the practice is backed up by empirical 
research that has been applied and show proof of 
ideas suggested by the study 


Setting up practices based on scientific research 




2 features sbr 


Validity 

Reliability 


Peer-reviewed 

Data-backed 




Define 1 type research 
des 


Case studies: follow people or situations over a 
specific period of time to record findings in 
empirical, real world settings that help describe and 
illustrate research and thinking 


Correlational studies: determining relationships 
between variables 




Diff exp & control group 


Exp gets a placebo and the real pill 
Control gets only the pill 


Exp: gets to test different variables 
Control: remains the same 




Random assign import? 


You are able to generalize to a larger population 


Fair and impartial 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Now use/like to use 
eviden 


6 


6 




How access research 


Internet, college/univ libraries, dissertations 


Same way: internet, checking with practices of 
other similar programs 




?s ask about partic study 


Is there a correlation between the results of the 
study and what you see in real world? 


Is it valid and reliable? 

Is it backed by strong data? 
Has it been applied? 




Key concept take away 




Much better understanding of scientifically based 








research principles and application 


3 


Define ebp 


Uses research evidence to complement the 
educational situation 


Teaching and learning based on evidence from 
research 




2 features sbr 


Lack of bias 
Double blind research 


Peer review 
Lack of bias 




Define 1 type research 
des 


Case studies: helpful but less scientific in approach 
True experiments: double blind studies where both 
the research subject and examiner are not allowed 
to know who is receiving the “new” product 


Case studies: soft data from questionnaires, 
classroom situations, testing 




Diff exp & control group 


The control group is not part of the subject 
population which receives something added to 
regular situation 


Exp group receives some added experience or 
product 

Control group is not receiving anything added 




Random assign import? 


Reduces the chance of bias in the research 


Essential for a non-biased research program 




Now use/like to use 
eviden 


3 


4 




How access research 


Internet 


Internet 




?s ask about partic study 


Does the research meet the criteria I am working 
with? Will the results be positive for my classroom 
population? 


Is there peer review? What is the method? 




Key concept take away 




The ability to see the importance of research (even 








soft data) and education 


4 


Define ebp 


Instructional practices that have been proven 
effective by research 


Integration of professional wisdom and 
scientifically based research to come up with best 
practices 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




2 features sbr 


Random sampling 
V alidity 


Validity 

Reliability 




Define 1 type research 
des 


Correlational studies: compare two dependent 
variables to see if there is a relationship between 
the two, such as higher IQ and higher SAT scores 


True experiments require random sampling, 
dependent variables and independent variables 




Diff exp & control group 


Control group is the “norm”; exp group is the one 
receiving “treatment” or a new method 


Exp group is the group that receives the 
“treatment”. Control group is the “normal” 




Random assign import? 


To make sure that groups are as similar as possible; 
to increase the internal and external validity 


To ensure validity 




Now use/like to use 
eviden 


4 


6 




How access research 


Reading articles on internet, Focus on Basics, 
articles provided by our department chair 


Flopefully, I will take more time to visit websites 
such as NCSAEE and NIEE to gain input 




?s ask about partic study 


Are the participants or groups similar to ours? Was 
it a true experiment? Was it valid? Was it an 
experiment that can be generalized? 


Are the results of the study generalizable to a 
broader population? Is it scientifically based? 




Key concept take away 




Research can inform and encourage us to pursue 








best practices. Thanks 


5 


Define ebp 


Practice based on prior past experience of what 
works and what doesn’t 


Practice based on available research and 
professional wisdom 




2 features sbr 


Validity 

Reliability 


Validity 
Reliablility 
Random sample 
Peer reviewed 




Define 1 type research 
des 


Case studies: studies of individuals 


Case studies: intense observation of individual case 
history 




Diff exp & control group 


Exp: try something different 
Control: stays the same 


Exp: make a change 
Control: keep the same 




Random assign import? 


Better makes the research apply to the general 
population 


To make it valid across the whole population 




Now use/like to use 
eviden 


2 


5 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




How access research 


Internet 


Internet 




?s ask about partic study 


Does it apply to our students/program? 


How does it apply to me/my students? 




Key concept take away 




Motivation to increase students’ retention 


6 


Define ebp 


Practices that have been researched and found to he 
factual 


A “practice” that has met all basic principles of the 
methods of scientific; proven by scientific method 
of research 




2 features sbr 


Non-hiased 

Controlled without anything to make it invalid 


Testable theory gives predictions that could be 
proven wrong 

Scientific knowledge has had to pass a test 




Define 1 type research 
des 


Case studies: using actual “real life” criteria 


Case studies: a sample study of real participants 
with specific criteria set 




Diff exp & control group 


Exp: group has variation in participation 
Control: no ifs, ands or huts: everything is the same 


Exp: given guidelines/extra guidance (group who 
practiced ball) 

Control: subjects given no help or guidelines 
(group who didn’t practice) 




Random assign import? 


A cross-section of the population is sampled, 
making the data reflect more than one segment of 
the population 


To get an accurate selection 




Now use/like to use 
eviden 


4 


4 




How access research 


On-line, educational magazines 


Internet, working with other CC 




?s ask about partic study 








Key concept take away 




Principles of scientific methods applied to 
research 


7 


Define ebp 


Research that has been validated and proven to 
work in the classroom 


Integrated professional wisdom with scientific 
based (valid and reliable) research to bring about 
positive results in the classroom 




2 features sbr 


Theory 


Systematic 

Rigorous 



22 




EvidEnceatxUtthe EirnLli v cfEss of EvidenceliBBed nacitice: A VUbrtehopIbrTisiring 
AdLit Basic Educ^Bory TAISF and One-Stop l^c«-UliotiEis and ftogarriActTirish^hnts 



Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Define 1 type research 
des 


Case studies: actual situations that have been 
written in narrative form for the purpose of 
discussion 


True exp: randomly selecting a group and 
comparing the overall results from an experiment 
of those who were provided extended practice and 
those who were not to validate the scientific theory 




Diff exp & control group 


Exp: randomly selected 

Control: selected specifically with much 

supervision 


Exp: given the support of additional practice 
Control: stays as is 




Random assign import? 


It provides more diversity to the study 


So there is a diverse group in which to base the 
scientific research. 




Now use/like to use 
eviden 


3 


4 




How access research 


Internet 


Using internet links given in this training 




?s ask about partic study 


Who was the research group? Who conducted the 
research? How long was the research conducted? 


Who were the participants? How long was it 
conducted? Is there other research available in that 
area? 




Key concept take away 




Be able to back up your professional wisdom with 
scientific research because it becomes more 
believable to others 


8 


Define ebp 


Using research (evidence that it works) to 
improve/enhance teaching (practice) 


Integrating professional knowledge with scientific 
research to make informed decisions 




2 features sbr 


Collect accurate data (unbiased) 
Interpret data correctly 


Random selection of control/experimental 
Systematic 




Define 1 type research 
des 


Case studies: study of a particular practice/student 
to see how it works 


Case studies: description of individual student or 
group 




Diff exp & control group 


Exp: get the new (“stuff’) instruction. 

Control: does not get the new stuff/instruction 


Exp: gets treatment 
Control; no treatment 




Random assign import? 


Unbiased 


Avoids biased results 




Now use/like to use 
eviden 


5 


4 




How access research 


Internet, journals 


Web, journals 




?s ask about partic study 


Is it from a reputable source? How alike is the 
research sample/situation to my situation? 


Is it from a reliable source? 
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/\ugiet20O7 



Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Key concept take away 




Importance of knowing where to find research and 








how to use it to improve instruction 


9 


Define ebp 


Using proven data to develop/inform program 
content and decisions 


Professional wisdom and empirical evidence 




2 features sbr 


Hypothesis 
Significant data 


Panel/peer evaluation 
Rigorous analysis 




Define 1 type research 
des 


Case studies: in-depth narrative and measurements 
that descrihe a particular organization/situation, 
etc. 


True exp: random sampling, pre and post testing 
and other empirical data; 1 variable (to the extent 
possible) 




Diff exp & control group 


Control group: does not have added intervention 
(or whatever the experiment is) so that you can 
compare what happens with and without 
intervention 


Control group doesn’t get intervention 




Random assign import? 


To have true data, you can’t select based on 
knowledge of individuals. Cohort must he widely 
representative and random sampling is most 
effective for that 


To ensure validity of findings 




Now use/like to use 
eviden 


4 


5: want to be informed but not a slave to research! 




How access research 


Relationship with ProLiteracy American, 
professional development opportunities (state and 
national conferences, SCALE) 


Start with our existing database. Have recently 
recruited grad student for this, will have better 
direction now. Contact NIFL and NCSALL. 




?s ask about partic study 


In what ways is the group studied similar to and 
different than our own organization? 


What factors made the difference? What were the 
results for the exper. Group vs. control? How 
replicable is it? 




Key concept take away 




Persistence study data: VERY useful! 


10 


Define ebp 


Strategies that have research-hased success 


Integration of professional wisdom with best 
available empirical evidence to deliver instruction 




2 features sbr 


Control group 
Study group 


Peer reviewed 
Replicated 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Define 1 type research 
des 


Correlational: proof that a strategy is aiding in 
student success, such as a class that requires 
supplemental instruction seems to have more 
student success than one without SI 


Case studies: highly detailed descriptions on how 
an instructional strategy operates in classroom 




Diff exp & control group 


The variable tested is withheld from control group 


Exp group includes intervention 




Random assign import? 


To provide diversity in the groups 


To control study variability 




Now use/like to use 
eviden 


4 


5 




How access research 


Internet and literature 


Internet 




?s ask about partic study 


Will this change provide student success and 
retention? 


Has the study been peer reviewed? Has the study 
been replicated? Is there a consensus in research 
community that findings are supported by 
additional studies? 




Key concept take away 




A program improvement plan 


11 


Define ebp 


Employing classroom instruction that is based on 
research 


Use of professional wisdom that is based on 
empirical research/combination of the two 




2 features sbr 


# of people studied 

Amount of time study done (period of time covered 
in study) 


Peer reviewed 
Data analyzed 




Define 1 type research 
des 


Case studies: discuss a situation and resolve the 
problems 


Case studies: various situations 

True exp: two groups, one controlled and one 

experimental under same conditions 




Diff exp & control group 


Control group is monitored more. More rules about 
who can make up the group 


Control group remains same and exp group 
received additional aids/training to see if there’s a 
difference 




Random assign import? 


It depends on what’s being researched 


Self selected participants may have a bias or pre- 
determined ideas 




Now use/like to use 
eviden 


4 


5 




How access research 


Mostly internet from various sources. Also books 
when preparing workshops 


Internet (have more sites to refer to), books 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




?s ask about partic study 


How many people were involved? How many 
variations were included? How long was the 
research carried out? 


Who participated? Were they randomly assigned? 
Was it reviewed by peers, and what peers? 




Key concept take away 




It’s not as difficult to incorporate research into our 
planning to help us meet our needs. 


12 


Define ebp 


You have evidence based on research that it is an 
effective practice with student success 


Research has been completed on and study groups 
done 




2 features sbr 


Helps to expand student knowledge 
Proven to he effective and helpful 


Has data to back it up 
Peer reviewed 

By investigating testable problems 




Define 1 type research 
des 


Case studies: where you get a certain group of 
people to experiment what you are trying to prove 
to he effective 


Case studies: you have information turned in and 
then a study was completed 




Diff exp & control group 


Control group is the one you know the end result 
Exp group to test unknown variables 


Exp group had time to practice. Control group no 
change. 




Random assign import? 


So you will get “accurate” end results (if not 
random, you will not get a true end result) 


So it is not fixed outcome 




Now use/like to use 
eviden 


4 


4 




How access research 


Internet, books, newsletters, etc. 


Internet, conferences 




?s ask about partic study 


If it is helping them or not 


Do you understand? What my outcome is, positive 
or negative 




Key concept take away 




Research can be helpful 


13 


Define ebp 


Practices that have been researched and have 
evidence 


Integration of professional wisdom with the best 
empirical evidence 




2 features sbr 


Individuals/subjects 

Study 

(illegible) 


Empirical findings 
Peer review 




Define 1 type research 
des 


Case studies: following an individual/subject in the 
research for results followed by narrative, more 
personal 


Case studies: more narrative 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




□iff exp & control group 




Control: random and not exposed 

Exp: subjects specifically chosen for the research, 

exposed 




Random assign import? 


Don’t have a biased group 


Prevent bias 




Now use/like to use 
eviden 


3 


4 




How access research 


Computer, books, articles 


Take you websites and begin searching 




?s ask about partic study 


(Hope I have better answers for the post tests!) 






Key concept take away 




The importance of research and its validity 


14 


Define ebp 


Practice (action-behavior-service delivery- 
instruction) that uses research findings (evidence) 
to direct it/support it 


Combination of empirical research and 
professional wisdom: practice of instruction, etc. 




2 features sbr 


Hypothesis 

Measurement-pre/post 


Peer review 
Random sample 




Define 1 type research 
des 


Correlational: studying variables in search of 
correlations, relationships, etc. Looking at two or 
more variables to gauge their interaction 


Correlational: pursue relationship of variable and 
outcome 




Diff exp & control group 


Exp: has variable(s) under study introduced 
Control: no variables controlled or introduced 
(“natural”) 


Exp: variable applied 
Control: no variable applied 




Random assign import? 


So that effects/results are not controlled or affected 
and therefore “muddy up” the results (keep sample 
“pure” and unbiased) 


To eliminate bias that affects (can effect) results 




Now use/like to use 
eviden 


3 


5 




How access research 


Prolit publications conferences/ncsall conf and 
training team accesses 


NCSAEE sites and links you provided 




?s ask about partic study 


Validity 

Relevance for our program 


Valid? 

Reliable? 

Scientific? 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Key concept take away 




Don’t be afraid of research. Look for what’s out 








there in areas of concerns/interests to us and then 
review for components that can inform and 
improve our work 


15 


Define ebp 


Using ideas/methods that have been tried hy others 
that have given results from their practices 


Employing professional wisdom with evidence 




2 features sbr 


Standardized rules 
Standardized equipment 
Fact based outcomes 


Professional wisdom 
Evidence from research 




Define 1 type research 
des 


Case studies: completed and reported on to 
enhance the (teaching or) use of information 


Case studies: study of a problem that was set to 
steps in order to overcome the problem 




Diff exp & control group 


Controlled is equal 
Experimental is random 


Exp: gets practice added incentives 
Control: does not get added guidance 




Random assign import? 


Because everyone learns differently. Therefore 
selecting research should cover all arenas 


It gives a truer results 




Now use/like to use 
eviden 


3 


4 




How access research 


Internet. Other instructors (colleges), pre-posting 
testing students 


Intemet/from other instructors/NCCCS office 




?s ask about partic study 


Is this practice helping our students to complete 
their education? Is this practice retaining students 
in the program? What could we do to improve 
upon this study and research for our program? 


Is it scientific research or non scientific? 




Key concept take away 




The action plan for our college for one of the areas 
of concern. 


16 


Define ebp 


Data obtained from studies and used as the focus or 
foundation of your teaching practice 


Wisdom and empirical evidence 




2 features sbr 


The variables used 
Length of study 


Control group 
Variables 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Define 1 type research 
des 


Case studies: to decide on what you are going to 
look at. Select the subjects to follow. Have a 
timeline for completion. Look to see if there are 
any common factors based on the established 
criteria/variables that were pre-determined 


Case study: study of a specific topic or group 




Diff exp & control group 


Control: the group you are trying to match 
Exp: the group that you are looking at to notice 
commonalities with the control group 


Control group: chosen to have no change in 
materials 

Exp: chosen to use new materials 




Random assign import? 


It gives a new perspective, opens up new questions 


Because it helps the validity of the research, it 
gives a truer picture of the group to be studied 




Now use/like to use 
eviden 


2 


2 




How access research 


Internet 


Internet 




?s ask about partic study 




Was it published? Has it been evaluated by peers? 
Is there any more research on the topic? Is this a 
new concept? Who was the population? 




Key concept take away 




Eimited adult ed research is available 
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Define ebp 


Practice based on research which has shown this 
approach to be effective 


Practice based on scientifically based research and 
professional wisdom 




2 features sbr 


Random assignment 
Control group 


Empirical methods 
Peer reviewed 




Define 1 type research 
des 


Case studies: studies of individual students 


Case studies: study of individual students 




Diff exp & control group 


An experimental group is being exposed to the area 
of research and the control group is not 


Exp group is exposed to variable and control group 
is not 




Random assign import? 


To ensure that there is no bias 


Important for validity and reliability 




Now use/like to use 
eviden 


5 


5 




How access research 


On-line websites and literature 


Recommended websites 




?s ask about partic study 




I would be interested to know if more than one 
study had been done 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Key concept take away 




Integrate scientific research with professional 








wisdom 


18 


Define ebp 


A practice based on research that has been tested 
using the scientific method to measure an outcome 


Use of strategy or method that you have utilized 
and found successful or useful 


2 features sbr 


Use of control groups 
Follows scientific method 


Control group 
Standard deviations 


Define 1 type research 
des 


Case studies: anecdotal accounts of real situations 
used for comparative purposes 


Case study: anecdotal account of a situation, what 
they were investigating and outcome without hard 
data for support 


Diff exp & control group 


Control: variables are manipulated 

Exp: uncontrolled variables: let’s observe and note 


Control: group that has manipulated variable 
Exp: random group 


Random assign import? 




To eliminate as many variables as possible 


Now use/like to use 
eviden 


4 


5 


How access research 


NCSALL, NIFL articles in print and on web. 
Additional links or references from these sources 


Web and journal articles 


?s ask about partic study 


Is this comparable to our student population and 
program and can we implement this? 


Did it follow scientific criteria and was it peer 
reviewed? 


Key concept take away 




Using reliable research as basis for implementing 
change 


19 


Define ebp 


As practicing methods/techniques that have been 
previously researched. The methods are evident 
that the research holds some validity 


The integration of research completed by experts 
that is utilized in the delivery of instruction 


2 features sbr 


Data and... 


Data 

Empirical findings 
Peer-reviewed articles 


Define 1 type research 
des 


Case studies: the study of a topic or issue over a 
certain period of time, the study comprises of 
various elements to be determined by the 
individual performing the study. However, the 
result is to examine if change has taken place. 


Correlational: testing whether there are links 
between variables and outcomes 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




□iff exp & control group 


Exp: not controlled by researcher 

Control: is a chosen group that the researcher 

controls 


Exp: is a group exposed to a new approach 
Control: group is not exposed to new approach 




Random assign import? 


Get better results 


You get a diverse group which allows one to 
examine the different variables presented in the 
sample group 




Now use/like to use 
eviden 


4 


4 




How access research 


Library and websites 


Library and web 




?s ask about partic study 




Does the validity or reliability decrease if the 
research isn’t scientific? 




Key concept take away 




Research is not as daunting as introduced in 








graduate school. 


20 


Define ebp 


Daily activities and program design informed by 
research 


Practice based on a combination of research and 
professional wisdom 




2 features sbr 


It begins with a thesis 
It tests the thesis 


Peer review 

Rigorous statistical analysis 




Define 1 type research 
des 


Case studies: evaluates a setting using primarily 
qualitative methods 


Correlational: evaluate the relationship of paired 
variables 




Diff exp & control group 


The experimental receives the method being 
evaluated while the control receives a method that 
may be typically used 


Exp: receive the trial activity or teaching style 
Control: receives the standard or established 
activity against which to compare results with the 
experimental group 




Random assign import? 


Research validity is based on a comparison to 
underlying variation in the population 


So that each group is equally representative of the 
total population 




Now use/like to use 
eviden 


6 


6 




How access research 


Internet, print 


Internet, print 




?s ask about partic study 


Is it relevant to my situation? Does it solve a 
problem in my setting/ 


Is it appropriate to my setting? Is the research 
valid? 




Key concept take away 




Evidence-based practice 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 


21 


Define ebp 


Instruction “practice” based on results of research, 
best practice, etc. of what is effective, results 
producing, and indicative of student and instructor 
growth 


Practice based on combination of professional 
wisdom, research findings 




2 features sbr 


Design based on accepted methodology 


Peer review 
Replicable 




Define 1 type research 
des 


True experiments carried out as a result of posed 
hypothesis, research of literature, structured design 
with outcomes measured against hypothesis 






Diff exp & control group 


Exp: typically is the “done to” group 
Control: no intervention related to research? 


Exp: done to 
Control: not 




Random assign import? 


Assures an unbiased or representative sample 


Replicates to extent possible the “target 
population” 




Now use/like to use 
eviden 


5 


6 




How access research 


Internet, publications, workshops, professional 
meetings 


Internet, workshops, journals, classrooms 




?s ask about partic study 


How are results similar to population and impact 
designed? What was research designed to do? 


Does it match criteria needed to improve practice? 




Key concept take away 




Integration of research to support best practices 
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Define ebp 


Teaching and program implementation that is 
based on research findings that are relevant to the 
program 


Employing professional wisdom with empirical 
evidence in making decisions on instruction 




2 features sbr 


Population used 

Instrument used to assess finding 


Research conducted meeting particular criteria 
Using a population assessed with statistics 




Define 1 type research 
des 


Research based on the study and assessment 
findings on specific, particular events, actions 


Research studies constructed with particular groups 




Diff exp & control group 


Control: group that is the norm 

Exp: group that is used to test and present new and 

different information or actions 


Control: the basic population group 

Exp: given new situation/matched against control 




Random assign import? 




Einds better, more accurate evidence 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Now use/like to use 
eviden 


3 


3 


How access research 


Pretest/post testing, internet, judgment of methods 
used to outcomes 


Plan assessment, testing 


?s ask about partic study 




Will this research improve my program or 
instruction? 


Key concept take away 




Scientific research can improve programs and 
teaching 
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Define ebp 


Using practices with 

instruction/retention/recruitment, etc. that has been 
proven through research to he effective 


A method that is used that has been researched and 
accepted by professionals in the area 


2 features sbr 


Quantitative data 
Pre-post assessments 


Peer review of articles 
Control group/experimental group 


Define 1 type research 
des 


Correlational: following two groups as they 
perform the same 

True: experimental and control groups 


Case studies: studying of a group of people or 

single person in context 

True exp: control group/experimental 


Diff exp & control group 


Exp: the one that is having something done to or 
changed 

Control: one that remains untouched 


Exp: something is changed, added, etc. 
Control: stays the same 


Random assign import? 


If you do not choose randomly, you will have 
biases and the research will not. . . 


If random assignment is not used, exp will be 
skewed because person may know something about 
subject or be familiar with or want to do it for 
some reason 


Now use/like to use 
eviden 


5 


6 


How access research 


Through staff development workshops, 
professional readings, and research done in classes 
for program use 


Internet, articles in journals, NCSAEE 


?s ask about partic study 


I would ask to see what %... 


Was it peer reviewed? 


Key concept take away 




Professional wisdom/empirical evidence 
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Define ebp 


Using results of research to guide practice 


No post-test data (left early) 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




2 features sbr 


Select theory or issue to study 
Sampling of universe (study smaller group) 






Define 1 type research 
des 


Case studies: look at in-depth descriptions of 
people, situations, etc., to make conclusions 






Diff exp & control group 


Exp: try something new with 
Control: do nothing different 






Random assign import? 


It is more like “real life”. You don’t get skewed 
results 






Now use/like to use 
eviden 


5 






How access research 


Internet, conferences, hooks 






?s ask about partic study 


Was the sample hig enough to he effective? Will 
the implementation he cost effective? Will it 
produce the desired outcomes/ 






Key concept take away 
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Define ebp 


No pre-workshop questionnaire completed (arrived 
late) 


Using informed instructional CR strategies so you 
read and judge research which you can use to make 
decision about what you do in the classroom 




2 features sbr 




Requires data analysis 
Multiple tools for collecting data 




Define 1 type research 
des 




Case studies: narrative which descrihes question, 
issue and what you did to address it 




Diff exp & control group 




Exp: intervention 

Control: conditions remain the same 




Random assign import? 




So you do not have as many variables 




Now use/like to use 
eviden 




6 




How access research 




I will look at statistics 




?s ask about partic study 




Who did the research? description of protocol? 




Key concept take away 




The need to keep sending research articles to staff 
and start discussion groups. 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 
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Define ebp 


No pre-workshop questionnaire completed (arrived 
late) 


Practice that is rooted in proven methods, not just 
common wisdom 




2 features sbr 




Clear research design, including questions, 
methodology, data analysis, results/findings 
Peer reviewed 




Define 1 type research 
des 




Case studies: qualitative research method involving 
close study/analysis of subject(s), often analyzed 
for themes or against qualitative findings 
True exp: exp and control group, introduces a 
dependent variable, random sampling, sample size 




Diff exp & control group 




Control group does not receive an 
intervention/dependent variable. It’s a baseline 
basis for comparison. The experimental group 
receives intervention; hypothesis is applied 
(illegible) 




Random assign import? 




Because self-selection biases the results of 
experiments by confounding results. Random 
sampling eliminates or reduces the bias people 
have when they choose to enroll in a study 




Now use/like to use 
eviden 




5 




How access research 




University and college libraries, Google scholar, 
professional publications (TESOL and NCSALL) 




?s ask about partic study 




Is it GENERALIZABLE? Is it reliable? Does it 
specifically address the needs of my program? Is it 
a well-designed study? 




Key concept take away 




That I’m not a “nerd” for always referring back to 
research. I’m glad NC supports this behavior. 
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Define ebp 


Program design, delivery, eval and modification 
based on research-derived evidence that is relevant 
to the program (continuous. . .process) 


Integration of empirical results from scientific 
research with “professional wisdom” in program 
practices 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




2 features sbr 


Formulate a hypothesis 
Use of controls 
Emperical observations 


Random sampling, hypothesis and testable design, 
quantitative analysis 




Define 1 type research 
des 


Case studies: assessment of particular population 
and time frame and study problem and results 
Correlational studies: attempt to relate 2 or more 
trends, characteristics, or outcomes — in relational 
results — not limited to cause and effect 
True experiments: testing a hypothesis 


Case studies: descriptive study of actual 
individuals, populations or events, designed to look 
at/answer “what’s going on here”? 




Diff exp & control group 


The control group is not subjected to the 
experimental regime but is assessed carefully by 
same measures 


Control and experimental groups are the same/have 
same characteristics but experimental group is 
subjected to changes/interventions and compared 
with control 




Random assign import? 


Avoids “self-selected” cases or populations; 
essential for statistical validity (a foundation) 


Avoid biases in the results 




Now use/like to use 
eviden 


4 


6 




How access research 


Google, prof, trainings, books and lit 


Same way (google, books/journals/prof, mtgs) 




?s ask about partic study 


Is it relevant, feasible to apply, result in improved 
outcomes? Plus resource prioritization/allocation 


Sample size, degree of statistical significance, 
methodology and limitations re: validity 




Key concept take away 




Refer to the websites more often for 
review/reinvigoration of evidence-based practice 
strategies 
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Define ebp 


Outcome by tasks studied 


Instructor experience/observation AND research 




2 features sbr 


Analysis, compiling results-evaluation 


Control group/experimental group 




Define 1 type research 
des 


Case studies: examples of studies in your area of 
interest 


LEFT BLANK 




Diff exp & control group 


A control group is part of an experiment 


One is acted upon; one is not. 




Random assign import? 


Cover larger segment for study 


Random says it all! No collection or grouping of 
like samples 




Now use/like to use 
eviden 


2 


5 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




How access research 


Staff workshops 


Journals, conference 




?s ask about partic study 


Participation success 


Why? Who are your subjects? How were they 
selected? 




Key concept take away 


Research the Instructor 

counts! 
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Define ebp 


The way of doing “something” which is 
defined/designed hy the results of studying the 
success and unsuccessful practices 


EBP has some science (research) behind it plus 
evidence of best practice 




2 features sbr 


Controlled studies, observations 


Data driven 




Define 1 type research 
des 


Case study: descriptive analysis of a situation 


Case studies: answers “what’s going on” 
Correlational studies: answers “what works” 




Diff exp & control group 


Experimental group is “acted upon”, while the 
control group is not 


Control group is not “acted upon” 




Random assign import? 


Prevents skewing results 


Prevents skewing data/results 




Now use/like to use 
eviden 


4 


5 




How access research 


Computer (google; library; literature; professional 
data) 


Pay more attention to journals; utilize links 
provided for suggestions 




?s ask about partic study 


Who conducted, what controls 


Was it published? Was it replicated? Supported by 
other research (significant amount)? 




Key concept take away 




Utilize the NIFE and NCSAEE websites 
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Define ebp 


Reviewing the literature, understanding the data, 
and implementing into current system. 


Taking one’s own wisdom and research-based 
material and implementing a goal-oriented plan. 




2 features sbr 


(1) A goal or directive for the research, (2) system 
to study the goal to produce outcomes 


(1) goal, (2) data 




Define 1 type research 
des 


Case studies-DFS; Correlational-Pa vloff’s dog; 
True experiments-drug studies 


Case studies-DFS 




Diff exp & control group 


The research is using control as the baseline 
(placebo); experimental do not know what is being 
given or studied 


Experimental group has an intervention; control 
group has no intervention 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Random assign import? 


Then the researcher doesn’t know who is receiving 
what and the sample doesn’t know. Truer data 
collection 


You have a variety of people: age, sex, knowledge 




Now use/like to use 
eviden 


Daily 


Daily 




How access research 


Internet, networking 


Internet, journals 




?s ask about partic study 


(1) Does the study give a good representation of 
the goal of our program? (2) How long ago was the 
study? (3) Does the study provide a good narrative 
about the findings? 


Is it relevant? What is the goal of the research? 




Key concept take away 




To keep reading and focused on making 
improvements. 
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Define ebp 


Teaching practices based on research 


Practices that are arrived at by practice, knowledge 
of the students, experiences 




2 features sbr 


Control group, hypothesis 


Control group, random sampling, consistency 




Define 1 type research 
des 


(No Answer) 


Case studies — following or studying several cases; 
correlational — studies that compare or find a 
correlation or connection 




Diff exp & control group 


A control group? 


Experimental group — changes are made and then 
compared to control group, which no changes are 
performed on. 




Random assign import? 


In order to get true and accurate scientific results. 


In order to not skew or contaminate the research 
outcome 




Now use/like to use 
eviden 


2 


(No Answer) 




How access research 


Seminars 


Websites 




?s ask about partic study 


Size of group or population study. How big was the 
study? 


Was the research a true experiment? 




Key concept take away 




Evidence-based research and empirical research 
are both important. Evidence-based, in education, 
is cheaper and probably very good in helping 
implementation. 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 
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Define ebp 


Teaching practices that are based on research that’s 
“proven” effective 


Practiced based on both research and own 
experiences 




2 features sbr 


Meta-analysis. Case studies have been performed 


Published? Replicated? 




Define 1 type research 
des 


Correlational — seeing how a variable change will 
create a drop or increasing in learning 


Correlational — seeing if a variable correlationally 
changes an outcome of a random sample 




Diff exp & control group 


Exp group includes the variable. Control group 
remains the same 


Control group doesn’t change variables. 
Experimental group does. 




Random assign import? 


We deal with “random” students in our classrooms 


No prior experience specific to the study is 
expected 




Now use/like to use 
eviden 


4.5 


4.5 




How access research 


Library, internet, books, other research 


W eb/periodicals/own/peer s 




?s ask about partic study 


Is what were doing working? Do we see positive 
changes? 


Has it worked for others? Does it make sense to 
our population? 




Key concept take away 




Sites/ideas from others/charged again 
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Define ebp 


Classroom instruction that mirrors what current 
research says works in a given curricular areas 


Professional knowledge and empirical evidence 
that drives instruction 




2 features sbr 


Meta-analysis. ? 


Peer Review, replication 




Define 1 type research 
des 


Correlational — studies that look at similar 
interventions or practices, compare and contrast 


Correlational— studies that compare what to what 




Diff exp & control group 


Control group — no intervention. Experimental — 
intervention 


Experimental — intervention. Control group — no 
intervention 




Random assign import? 


It gives “true” results. Prevents “setting up” a 
group to succeed or fail. Anonymity. 


Takes away bias, gives more “true results”. 




Now use/like to use 
eviden 


3 


4 + 




How access research 


Web, library, UW 


Connect research to school improvement goals — 
GREAT resources given!! 




?s ask about partic study 


Who, what, why, results 


Who is being studied, why? What are the research 
questions? Outcomes? Implications? 




Key concept take away 




Connecting research to data leading to effective 








intervention 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 
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Define ebp 


Research that proves or disproves the use of 
instructional strategies 


Professional experience + research-based practices 
= interventions and strategies 




2 features sbr 


Hypothesis. Conclusion based on data 


Experiment, hypothesis, conclusion 




Define 1 type research 
des 


Case studies — track individual. Correlational — 
compare two studies. True experiments — have 
experiment and control group 


Case studies — individual studies. Correlational — 
discover external, internal influences in a study. 
True experiments — control and experimental 
group. 




Diff exp & control group 


Exper — change is introduced. Control — status quo. 


Exper — implement a change. Control — no changes 




Random assign import? 


Reduces bias in data 


Eliminates bias 




Now use/like to use 
eviden 


5 


5 




How access research 


Conferences, internet, books 


Internet: provided on handout. Books 




?s ask about partic study 


(1) what is the agenda of the research? (2) size of 
group? (3) valid conclusions? (4) where published, 
(5) studies to support this one? 


(1) type of study, (2) results, (3) group, (4) # in 
study 




Key concept take away 




Improved knowledge in analyzing data and 
research. 


35 


Define ebp 


How things are done in classroom using empirical 
evidence gathered by legitimate sources 
concerning best ways to present materials 


Integrate professional wisdom with good, empirical 
evidence to make decisions on material delivery 




2 features sbr 


? 


Random sample, control group 




Define 1 type research 
des 


Correlational — do your research and compare to 
similar studies 


Correlational — outside studies for comparison 




Diff exp & control group 


Exper — changing way of delivering services. 
Control — maintain current standards of delivery 


Experimental — experience the changes being 
incorporated. Control — everything stays same 




Random assign import? 


So as not to hand pick clients which could skew 
results 


So sample will not be skewed by a certain 
characteristic 




Now use/like to use 
eviden 


3 


3 




How access research 


? I don’t think we do. We let Betty do it. 


Use web sites, journals 




?s ask about partic study 


Who? When? What? Where? Population make up 


Who, what when, where, why? 




Key concept take away 




Use “(illegible) look” of wisdom being presented 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 
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Define ebp 


To use what has been professionally proven 


Something I may use 




2 features sbr 


Scientific studies over a period of time with 
changes in study to prove facts 


Number. Proven 




Define 1 type research 
des 


True experiments will use a new drug (vs) placebo 


Case studies are watched and monitored. 




Diff exp & control group 


Control group will have guidelines. Experimental 
will figure out a way to make it work 


Control group needed to practice. The 
experimental group skunked them after practice. 




Random assign import? 


It’s unknown. However, many people will have the 
mental feeling of wellness simply by suggestions. 


Keeps the odds even. 




Now use/like to use 
eviden 


5 


5 




How access research 


Internet, educational outreaches 


Internet or educational facility 




?s ask about partic study 


What are the proven facts? 


Outcome, who are researchers 




Key concept take away 




I can help to prove my thoughts. 
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Define ebp 


Teaching based on research 


Based on research, professional wisdom, 
experience 




2 features sbr 


Methodology and replication 


Research question, data 




Define 1 type research 
des 


Case studies — looking at individuals whole self 
before, during and after applying or using a 
variable to create change 


Correlational — telling what is going on 




Diff exp & control group 


Control group is the comparison group. One would 
use after applying a variable to an experimental 
group 


The experimental gropu has an intervention done 
to it; the control group does not. 




Random assign import? 


A better generalization of the population as a 
whole can be made with the outcome. Accounts for 
a variety of factors with less bias from researcher. 


Random assignment accounts for limitations in the 
general population and “evens the odds”. 




Now use/like to use 
eviden 


4 


5 




How access research 


Google, Focus on Basics, MPAE A journal 


Through all the websites and their links 




?s ask about partic study 


Who — the researcher and participants. What — 
what is the variable and action done to the 
participants? 


Is it valid, is it relevant to our situation, did that 
research suggest program help or improvement? 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Key concept take away 




We’re doing things right, research is easily 








accessible on the web. 
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Define ebp 


Evidence-based practice is use of research proven 
strategies to improve instructional practices 


Professional wisdom combined with scientific 
evidence on how to deliver instruction 




2 features sbr 


Proven research with case studies — 
results/outcomes. Control group as defining group 


Believe that new evidence critical to work 
effectively with students. Integrate what have 
learned with previous knowledge 




Define 1 type research 
des 


Case studies — specific study with hypothesis, 
procedures and outcomes 


Case studies: detailed description of individual or 
small groups: “What’s going on?” 




Diff exp & control group 


Experimental group — large group. Control 
group — specific cohort (average diversity) 


Experimental group learns through experience and 
practice. Pre- and post-assess to determine gains as 
result of practice. Control group is static group — 
not influenced. 




Random assign import? 


So that “truer” results are achieved across a 
population 


Gives a better cross-section of general population 
without bias or pre-conceived experience. Truer 
results. 




Now use/like to use 
eviden 


3 


3 




How access research 


NIEE, NCSAEE, EINCS, conferences 


NICE, NCSAEE, EINCS, abstracts/research 




?s ask about partic study 


Eeasibility, practicality, and demonstrated 
outcomes 


(No answer) 




Key concept take away 




Use evidence-based research and practice to guide 
program improvement 
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Define ebp 


Research based on actual practice 


Implementing practices which are based on both 
research and teacher knowledge 




2 features sbr 


(1) statistics, (2) examination and/or inclusion of 
other research 


(1) control group vs. experimental group, (2) stats 




Define 1 type research 
des 


Case studies — ^research designed around and with a 
specific set of criteria (i.e., with a control group) 


Case studies — examines/studies a particular 
group/individual and reports on observed 
characteristics 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




□iff exp & control group 


Control group is a “group” in which the 
experimental group results are compared to' 


Experimental — group which 
“experiences’Vparticipates/tests hypothesis. 
Control — group which “maintains status quo”. 




Random assign import? 


Because it allows for a truer “picture” 


Because it provides for a wider 
distribution/comparison to the actual population 




Now use/like to use 
eviden 


4 


4 




How access research 


Through the internet and journals 


Utilize links given 




?s ask about partic study 


How the research was set up: methodology used; 
statistics: was random sampling used? 


Was it published and recognized? Was it 
replicated? 




Key concept take away 




Utilize concepts/websites, etc., given 
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Define ebp 


Practices that have been proven to work 


Scientifically based research using scientific 
method 




2 features sbr 


Scientific method; follow up 


Practice; control group 




Define 1 type research 
des 


Case studies: numerous comparisons about the 
same topic 


Case studies: look at different studies to see if a 
correlation can be made about the outcomes. 




Diff exp & control group 


Experimental: use of the e xperimental object. 
Control: not using the thing, given a placebo 


Control group doesn’t use any remediation or 
change. Experimental group practices or changes 
something 




Random assign import? 


So that the research is not biased 


There is no bias; the group is better representative 
of the population 




Now use/like to use 
eviden 


3 


3 




How access research 


Reading, computer 


By looking at other research out there and trying 
new methods 




?s ask about partic study 


Outcomes? Cost. Ease of use 


How long has this practice been in use; findings? 




Key concept take away 




Ways to implement better reading strategies 
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Define ebp 


A practice that is developed based on research 


A practice based on research or “evidence” that has 
been tried/proven 




2 features sbr 


Valid. Tested 


Validity and reliability 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Define 1 type research 
des 


Case studies are research based on a specific item 
or group. The study would have a research 
question and then explain the process used 


Correlational studies compare the effect of 
different “interventions” on a subject 




Diff exp & control group 


The experimental group is exposed to whatever 
you’re researching. The control group is not 
exposed. 


The experimental group receives an intervention. 
The control group does not. 




Random assign import? 


Random assignment helps to keep the results more 
valid. Someone could “stack the deck” if they 
“choose” the sample 


Allows for a more accurate representation of the 
population 




Now use/like to use 
eviden 


4 


5 




How access research 


Journals, meetings, and the internet 


Through specific websites/journals 




?s ask about partic study 


Is it applicable to my students? Is it practical (in 
terms of cost, time, outcomes)? 


Are the results valid, reliable and practical? 




Key concept take away 




That research can be valuable. 
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Define ebp 


Putting into practice, implementing procedures that 
have been proven to work 


Integration of research and professional wisdom. 
Then make decisions about services and practice. 




2 features sbr 


Been implemented in several settings. Placebos 
have been used so that research wasn’t 
manipulated. 


Been researched in more than 1 research. 
Published. 




Define 1 type research 
des 


Case studies: look at particular groups and 
gathering information 


Case studies: study specific case; not broad. 




Diff exp & control group 


Experimental: the variables aren’t taken into 
account. Where control they try to account for 
variables that might be present. 


Experimental implement the change to compare to 
the control group. 




Random assign import? 


If not random selection, you could prove anything 
by manipulation 


So can’t manipulate outcomes. 




Now use/like to use 
eviden 


3 


3 




How access research 


Internet: literacy materials and information. 
Training, conferences. From ABE/GEd leaders and 
industry 


NCSAEE, NIFE, CAE, TESE, ETS.org 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




?s ask about partic study 


How many groups and where were the groups 
studied that “prove” this works? 


How many projects/people have proven this to be 
true? 




Key concept take away 




Websites with research. 
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Define ebp 


Praetice that is based on good research and proven 
to get the results needed 


Integrates professional knowledge with research 
practices to determine the best way to do things. 




2 features sbr 


Statistical analysis. Reliahility. 


Reliability and validity 




Define 1 type research 
des 


Case studies: study individuals and how they 
respond to programs/intervention 


Case studies: study of indiv to determine effect of 
practices on outcomes 




Diff exp & control group 


Control group is randomly selected and does not 
receive/participate in intervention. Experimental 
one does. 


Exp: participants in intervention. Cont: doesn’t part 




Random assign import? 


So the study is not biased and provides better data 


Gives you an unbiased selection 




Now use/like to use 
eviden 


5 


5 




How access research 


Journals, internet, Dept Ed 


Same way 




?s ask about partic study 


Was group similar to mine? Was the study 
conducted in a reliable scientific manner? Is it 
something we can mimic and implement with 
success based on what we know now. 


How was it done and does it apply to corrections? 




Key concept take away 




Mean, media and mode! Using research better to 
help me improve program delivery. 
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Define ebp 


A practice that has been put into use because 
research has shown that it works. 


Combination of empirical evidence and 
professionalism in practice. 




2 features sbr 


A control group. A need for the type of research 
done. 


Published in an accredited journal. Have results of 
study been replicated by other scientists. 




Define 1 type research 
des 


Case studies: looking at groups in their natural 
settings to do the research 


True experiments: randomized tests that answer 
what works 




Diff exp & control group 


A control group is a group where no new practices 
are applied or tried, whereas an experimental group 
is subjected to different or new practices. 


A control has no outside influences whereas an 
experimental group has some new or different 
aspect to it. 




Random assign import? 


So that there is no bias 


So that there is no bias or predisposition 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Now use/like to use 
eviden 


1 


3 




How access research 


N/A 


Online 




?s ask about partic study 


Whether the results of the research are significant 
and if they apply to my situation. 


Whether there is a useful aspect from the results of 
the research 




Key concept take away 




That research can make a difference in 
instructional practices 
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Define ebp 


Statistics that show what really works in 
classrooms 


Promotes teaching/leaming in a fashion that has 
been tested. 




2 features sbr 


Outcome. Using same text/time factor 


Research that is credible and useable. 




Define 1 type research 
des 


Correlational studies: determine how different 
types correlate as studies unfold 


Correlational: collecting data, studying outcomes 
and connecting the data for instructional practices 




Diff exp & control group 


Experimental: all parties know. Control group: one 
group is being ohserved/one isn’t. 


Exp: performance with no control. Control: is 
controlled by instructor. 




Random assign import? 


Yes, because of fairness. Less bonding, less 
becoming emotionally involved 


Random sampling addresses a more accurate 
conclusion while. . . 




Now use/like to use 
eviden 


4 


5 




How access research 


Based on past successes 


Research, evaluate and apply to classroom setting 




?s ask about partic study 


How does one conclude the true learning success 
of non-English speakers? 


Would this be applicable to our region? How might 
we implement this with the time factor of part-time 
teachers? 




Key concept take away 




Involvement-interaction with groups. 
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Define ebp 


Practice that has been proven to work 


Using personal wisdom with empirical evidence to 
make decisions 




2 features sbr 


Surveys. Control group. Eactual data. 


Systematic. Rigorous data. 




Define 1 type research 
des 


Case studies: people. Information on individuals 
recorded and studied over a period of time. 
Correlational: comparison of case studies. 


Case studies: individual/group descriptions. 
Correlational: tests links between variables and 
outcomes. True experiment: randomized trials that 
answer “what works” questions 




Diff exp & control group 


Control group: specific gender, age, etc., 
regimented 


Experimental group is random group 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Random assign import? 


Random assignment will include variables to. . .(not 
finished) 


Needed for comparison 




Now use/like to use 
eviden 


Left blank 


4 




How access research 


Internet 


Internet with specific given addresses: ncsall.net, 
NIFL 




?s ask about partic study 


Witt this change be relevant for my students? 


Size of group, gender ,age 




Key concept take away 




Use common sense in applying research to 
students. Use what is relevant to them or what is a 
“good fit” to help students improve. 
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Define ebp 


Practice based on evidence/research 


Practice based on research 




2 features sbr 


Well researched — based on, grounded in previous 
research/plus actual happenings. Tries to remove 
research biases. Findings similar to other studies. 


Same answer. 




Define 1 type research 
des 


Case studies: individual studies, thorough findings 
based on correlations, can’t show cause and effect 


True experiments: based on scientific method, 
control on variables, and strongly based on former 
research 




Diff exp & control group 


Exp: receive the orange drink/poison. Control 
group: does not receive the orange drink/poison 


Exp: receives treatment. Control: does not. 




Random assign import? 


Minimize bias 


Minimize variables/other factors 




Now use/like to use 
eviden 


2 


4 




How access research 


No. We had really the best workshop recently that 
brought in research 


Adult ed research/NCSAEE 




?s ask about partic study 


Try to find out how similar the exp group is to my 
population and conditions. Also how sig finding 
are. 


Flow similar population is to my population 




Key concept take away 




Research locations: where to find it. 
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Define ebp 


A practice based on evidence which would be a 
step further than the “research” based 


Personal wisdom and best available evidence. 
“What should drive practice” 




2 features sbr 


Needs lots of time. Needs lots of participants. 


Systematic. Rigorous data. 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Define 1 type research 
des 


Case studies: studies of particular cases for a 
particular reason/research 


Case studies: “What’s going on” - detailed 
description of individual/smah groups 




Diff exp & control group 


Exp: not assigned. Control: assigned. 


Exp: random. Control: not random 




Random assign import? 


So you can get a more accurate reading 


You get a better cross-section. 




Now use/like to use 
eviden 


2 


3 




How access research 


Left blank 


Web; given sites 




?s ask about partic study 


In what particular cultures does this seem to work? 
Age group? Socio-eco? 


What were the findings? Will they work with/in 
my areas? 




Key concept take away 




Left blank 
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Define ebp 


Practice that is conducted which is based upon 
current research findings 


Practice that grounded in the theory of the field. 




2 features sbr 


Use of inductive research. Use of deductive 
research 


Has a question that is researched for an answer. 
Involves the systematic review of literature. 




Define 1 type research 
des 


Case studies: qualitative methodology where a 
group is studied according to the parameters 
established by Merriam, Lincoln and Denizen. 
Includes triangulation for the establishment of firm 
controls and gives credence to findings. 


Case study: study of a collection of subjects for the 
study of thoughts, feelings and perceptions in a 
structured format (Lincoln, Cuba, Merriam, 
Denizen) 




Diff exp & control group 


Exp: has treatment performed on them. Control: no 
treatment 


Exp: receives the treatment. Control: does not 
receive treatment. 




Random assign import? 


Random assignment is important to establish the 
measurable variable. I also believe that a 
purposeful sample is valid for case study research. 


The results are more likely to reflect the population 
sample. 




Now use/like to use 
eviden 


4 


4 




How access research 


Books and periodicals in the field, research reports 
form NCSALL and NILE, National on-line 
discussion lists, discussion with others in the field. 


Through books and periodicals in the field, online 
searches, professional organizations, national 
listserves, TCALL, NCSALL, NILE, LINCS 




?s ask about partic study 


Is this something that can help the teachers and 
students with in my region? 


What will the benefit be to the students and/or 
teachers? 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Key concept take away 




Better understanding of how to get a pilot research 








project started and subsequent follow-up. Thanks! 
Great workshop. 
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Define ebp 


Evidence-based practice is the use of things that 
“work” for one and have been show (evidence) to 
“work” 


Same as before 




2 features sbr 


You have an experimental group and a control 
group. In a controlled setting. 


Same as before 




Define 1 type research 
des 


Case studies: provide documentation of research 
from start to finish. Correlational: study whether 
this variables covary with each other. True 
experiments: (science based) contain a control and 
experimental group in a controlled setting are 
closely monitored. 


A correlational study is a study that provides 
evidence that two variables co-vary together. 




Diff exp & control group 


Exp: is introduced the independent variables. 
Control: is not introduced to the independent 
variable. 


Whether you introduce the independent variable or 
not. 




Random assign import? 


It is important because you do not want your 
research sample to be contrived. You really want a 
true representation of your population. 


To be able to be sure your sample is representative 
of the population and generalizability 




Now use/like to use 
eviden 


5 


5 




How access research 


Through various resources- library, TCAEE, 
various databases 


Same as before: TCAEE, NCSAEE, etc., and 
Academic Search Premier 




?s ask about partic study 


Were the findings significant? Were there any 
significant findings relevant to my particular 
study? 


Is it a true research-based study? Are the findings 
valid? 




Key concept take away 




Accessing, understanding, judging and using 
research. 


51 


Define ebp 


Research that has been applied and evaluated in the 
classroom 


Practice based on solid evidence that it will work. 
Evidence gathered methodically and analyzed 
appropriately. 



49 




NCSALL Occasional Ffeper 



/\ugiet20O7 



Resp. 


Question 


Pre-Workshop 


Post-Workshop 




2 features sbr 


Identify an area of investigation. Apply findings to 
real situations. 


Left blank 




Define 1 type research 
des 


Case study: observation a nd study of a specifie 
research situation 


Case study: indepth study of a particular subject 
(individual) 




Diff exp & control group 


Left blank 


The experimental group receive the intervention 
strategies. 




Random assign import? 


Left blank 


To get a true picture of the reality. 




Now use/like to use 
eviden 


4 


3 




How access research 


Through publications, internet, conference 
presentation, and from researchers in the field 


Through the various alphabet soup sites: TCALL, 
NCSALL, NIFL, etc. 




?s ask about partic study 


Who conducted the study, how was it 
implemented, what were the results, what follow- 
up was done after the study was completed, will 
there be ongoing study? 


Is it pertaining to what I am trying to achieve? 




Key concept take away 




Left blank 


52 


Define ebp 


Teaching practice which a foundation in research 
theory (sic). Teaching based on what research has 
demonstrated “works”. 


Practice based on research and professional 
wisdom both. 




2 features sbr 


That it is objective and based on verifiable data. 


Random sampling. Reviewed by a peer journal and 
accepted as viable. 




Define 1 type research 
des 


Case studies: studying one segment of a field, for 
example, one adult ed program in order to learn 
more about what practices work. 


Case studies: detailed study of a limited group of 
people can be quantitative or qualitative. 




Diff exp & control group 


With a control group, variables are fixed and you 
know exactly who you are researching. With 
experimental, there is no known “control” to base 
your variances on. 


Exp group is “put to the test” and then results are 
compared to control group who is not being 
evaluated. 




Random assign import? 


So as to not skew or favor an outcome. Random 
assignment ^objectivity 


For purposes of objectivity and unbiased dta. 




Now use/like to use 
eviden 


2 


Left blank 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




How access research 


Reading materials 


Left blank 




?s ask about partic study 


Who were the participants and how many were 
there? 


Left blank 




Key concept take away 




Left blank 


53 


Define ebp 


NA 


NA 




2 features sbr 


NA 


NA 




Define 1 type research 
des 


Case studies: the life and experience of a followed 
individual within a time frame, whether it is a short 
or a long time frame. Correlational: two factors are 
evidence and reveal a correlation that is parallel 
and impacts one another. 


Same as before 




Diff exp & control group 


Experiment is not an exercise; it is usually an 
empirical approach that monitors cause and effect. 
A control group is a group is exposed (sic) and has 
no tampering with material while other group is 
exposed and a certain level of control is 
maintained. 


Whether you introduce an independent variable or 
not. 




Random assign import? 


So that you can have a healthy range of 
participants from various backgrounds impact the 
research instead of a specific gender, ethnic group, 
educational exposed. This instead allows an 
accurate range of reality-hased evidence. 


It takes a range of various backgrounds and places 
them in an experiment that can be controlled or 
uncontrolled/blind or double blind. 




Now use/like to use 
eviden 


6 


6 




How access research 


I use empirical research. 


There are a number of different websites out there 
that we can look and evaluate information from. 




?s ask about partic study 


NA 


Can it be replicated? Is there sufficient evidence 
that support these findings or can it be researched 
further? 




Key concept take away 


What are you to how to he 

implemented, and understanding/assessing. 
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NCSALL Occasional Ffeper 



Augiet2G07 



Resp. 


Question 


Pre-Workshop 


Post-Workshop 


54 


Define ebp 


Practice in teaching where choices for instruction 
are founded in evidence of effectiveness for 
students. 


Same as before 




2 features sbr 


Theory. Control groups. 


Control group. Pilot projects. 




Define 1 type research 
des 


Case studies detail the background and experience 
of an individual participating in research. 


Same as before 




Diff exp & control group 


Exp: tries the new procedure. Control group 
follows procedure practices previously. 


Same as before 




Random assign import? 


So that a research sample represent as wide a range 
as possible of student characteristics, needs. 


Same as before 




Now use/like to use 
eviden 


3 


4 




How access research 


Through professional development workshops 


Internet websites. Thank you for these! 




?s ask about partic study 


Whether participants were native speakers of 
English. Whether results of the study had impact 
on retention. 


Thank you for handout on evaluating research. 




Key concept take away 




The availability of research findings on the 








internet, as with www.ncsall.net. Thank you for a 
very informative, well-organized presentation. 


55 


Define ebp 


Practicing or re-defining your actual practice in the 
classroom based on evidence we discover to meet 
the students’ needs. 


Working to make changes through research 
evidence. 




2 features sbr 


Eeft blank 


It is well documented. The sample group is large. 




Define 1 type research 
des 


Case study: looking at all aspects of a particular 
situation. 


True experiments are like seeing if the little ball 
can be popped into the cup. 




Diff exp & control group 


A control group is a separate group from the actual 
one we are studying to help us maintain 
perspective. 


Eor purpose of comparison. 




Random assign import? 


So as not to be personally prejudiced as a 
researcher 


So there is no bias. 




Now use/like to use 
eviden 


2 


4 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




How access research 


Not on a formal basis but we as teachers are 
constantly evaluating and making changes. 


I will look at things differently, thinking how we 
can improve. Research can help us. 




?s ask about partic study 


Is it applicable to our situation? 


Accepted by colleagues (published). 




Key concept take away 




There is a lot of research out there that can help us 
do a better job, if we will use it. 


56 


Define ebp 


Practice that results in valid and reliable data that 
support or represent change in student 
skills/knowledge/abilities 


Practice based in empirical research 




2 features sbr 


Replicable. Valid. 


Replicable. Valid. 




Define 1 type research 
des 


Case study: based in qualitative data and represent 
actual or couples. Correlational: that relate data 
from one group to that of another. 


Case study: qualitative data 




Diff exp & control group 


Experimental gets the “treatment” exposure. 
Control gets status quo. 


Exp: treatment. Control: no treatment 




Random assign import? 


Depends on type of research. If an experimental 
design, it tends to assure that results are due to 
some intervention other than chance. 


Allows researchers to lower probability of change 
driven by variables other than the treatment or by 
chance. 




Now use/like to use 
eviden 


4 (but I’m primarily a qualitative researcher) 


3 




How access research 


Peer reviewed journals, research reports. Listserve 
announcements/discussions 


Same as before 




?s ask about partic study 


Did the methodology choice math the research 
question? How was the methodology process 
applied? How were the data analyzed? Did the 
researcher draw reasonable conclusions from the 
data? Are the results/conclusions/generalizations 
applicable to my practice? 


Same as before 




Key concept take away 




Good idea to use in my grad intro to research 
analysis course. 


57 


Define ebp 


Something that has supportive evidence to back it 
up. 


(Wrote “same as before” on the top). True 
experiences and highly detailed 




2 features sbr 


Demographics. Size of sample. Controlled or not. 


Evidential research. Control groups. 
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/\ugiet20O7 



Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Define 1 type research 
des 


Correlational studies: studies conducted to prove a 
theory. 


True experiments: done with a control group and 
an intervention group. 




Diff exp & control group 


Exp: an open-ended study. A control group is 
followed all the way through a study. 


Control continues same mode. Experimental has 
intervention. 




Random assign import? 


If you pick your assignment then results may he 
more biased. 


Because it is less likely to be biased. 




Now use/like to use 
eviden 


4 


5 




How access research 


Internet, TCALL, PP 


NCSALL, TCALL websites etc. 




?s ask about partic study 


Learning disabilities. 


Does it work? How was it done? 




Key concept take away 




Group work. Exercises 7.2 


58 


Define ebp 


In reference to GED programs (one of which we 
have), it means a practice that we use because we 
have seen good fruits from in the past. 


Scientific basis for the practice. 




2 features sbr 


A control group and a group you change modes on 


Assessment. Judgment 




Define 1 type research 
des 


Correlational: a study that illustrates something 
like this: studies have show there is a direct 
correlation between the # of people who eat 
oatmeal regularly and who die within 20 years. 


Correlational: certain aspects of a group 
correspond to other aspects. 




Diff exp & control group 


The control group have not had unusual change 
modes applied to them. 


One has change introduced; the other does not 




Random assign import? 


If you select a related group, you run the risk of 
error due to factors that relate them 


Cleaner data or results 




Now use/like to use 
eviden 


4 


5 




How access research 


In-house mostly. 


According to how it is collected 




?s ask about partic study 


How does it apply to your program? 


Is it case studies, correlation, true exp? 




Key concept take away 




How to assess and collect research. 


59 


Define ebp 


The application of a practice based on tested and 
proven effectiveness of this practice 


The implementation of practices in your program 
based on research that demonstrates or 
substantiates its efficacy. 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




2 features sbr 


Must not be biased. Must be based on a large 
enough random sample to support conclusion. 


It is structured and systematic 




Define 1 type research 
des 


Correlational: evidence indicates strong support for 
a conclusion based on cause and effect. 


Correlational studies: answers the question, what 
are the similarity or differences? 




Diff exp & control group 


Exp: tbe cohort actually tested. Control: the cohort 
not in test but used for comparison with the 
experimental cohort 


Exp: tests the new idea or practice by putting it into 
effect. Cont: does nothing. 




Random assign import? 


To exclude bias from the results 


It keeps the results “clean” 




Now use/like to use 
eviden 


3 


4 




How access research 


ERIC articles, NCAEE and NIEE and my own 
research 


ITl go to websites: NCSAEE, NIEE, TESOE, 
CAEEA 




?s ask about partic study 


Does the suggested practice fit into our program? 
Is the suggested practice compatible with what we 
want to do? Is it at cross purposes in any way with 
our prog purposes and goals? Does it help or 
hinder? 


What does professional wisdom indicate we ought 
to do or not do? 




Key concept take away 




Handout 7.2 is very useful! 


60 


Define ebp 


Evidence-based practice is based on aligning 
teaching with research. 


Instruction aligned with research 




2 features sbr 


Validity. 


Reliablity. Validity 




Define 1 type research 
des 


A case study explores an actual individual or group 
of individuals involved in a controlled research 
study. 


True experiments: answers “what works” questions 




Diff exp & control group 


Exp: variable, changes. Control: stays constant, no 
variables 


Control: no variable. Exp: variable 




Random assign import? 


It allows for a valid sample of subjects. 


Validity 




Now use/like to use 
eviden 


4 


5 




How access research 


Eric. Take classes. Member IRA 


ERIC, school, TESOE 




?s ask about partic study 


Is it applicable to my students and their needs? 


Is it applicable to my students? 




Key concept take away 




Align teaching with research 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 


61 


Define ebp 


Putting into practice something that is proven 


Basing what you do (practice) on proven research 




2 features sbr 


Must follow a specific “method” 


Validity. Reliahility. 




Define 1 type research 
des 


Case studies: researching “real life” events/people 
and reporting the findings 


Case studies: answers “what’s going on” 




Diff exp & control group 


Exp group lacks pre-arrangement modifications 


Control group must follow different guidelines 
under more control 




Random assign import? 


Must include all types of the sample 


So as to have validation 




Now use/like to use 
eviden 


6 


5 




How access research 


Internet 


Different weh sites 




?s ask about partic study 


Who was the researcher? 


Who was asked; what were findings 




Key concept take away 




Research is important and should he incorporated 
into instruction along with past experience 


62 


Define ebp 


Something that you do “a practice” because you 
see that it works (successful) 


Integration of professional wisdom with the best 
available empirical evidence in making decisions 




2 features sbr 


Experimenting with a new way of doing 
something. Coming up with ideas or new ways to 
do something. 


Research and conclusions 




Define 1 type research 
des 


Correlational: studies done with a group of peole 
that have certain needs and how your idea relates 
to their needs 


Correlational: test links between variables and 
outcomes that answers “what influences what”? 




Diff exp & control group 


A control group doees not change or do anything 
new or different than previously. The exp group 
tries new ideas and you can compare the results to 
the control group 


Control group: doesn’t change. Exp group 
undergoes a change. 




Random assign import? 


This gives you a true outcome of how a new 
project would affect most other programs. 


You get a true group that represents a general 
population. 




Now use/like to use 
eviden 


2 


5 




How access research 


If something worked and students are successful, 
then I do it again. If not, I don’t. 


Utilizing the different sites. 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




?s ask about partic study 


What are the students doing now? Did this help 


Who does this help? How does it help them? What 






their family also? 


is the cost? 




Key concept take away 


Pay more to 

let other research help my program. 
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Define ebp 


Something you do that is successful 


The integration of both empirical evidence and 
professional wisdom in making decisions 




2 features sbr 


? Experiment 


Employ systematic, empirical methods that draw 
on observations. Involves rigorous data analysis. 




Define 1 type research 
des 


7 


Case studies: highly detailed descriptions of 
individuals or small groups that answer “what’s 
going on” 




Diff exp & control group 


Exp=you experiment with unknown variahles. 
Control=experiment with known variahles (you 
control the variahles) 


Exp group has interventions and control group 
does not. 




Random assign import? 


? you have different people doing the same work 
and getting the same answer. 


Different people, different backgrounds 




Now use/like to use 
eviden 


1 


5 




How access research 


7 


Visit the different sites that were given to us. 




?s ask about partic study 


7 


Eeft blank 




Key concept take away 




SSR 


64 


Define ebp 


Instructional practices which may not have 
scientifically based randomized studies behind 
them, but at least have some systematic research 
and are widely accepted by researchers in adult 
education 


Instructional or programmatic changes based on 
scientifically based research or best available 
research supported by community of researchers in 
field. 




2 features sbr 


Earge, robust samples. Random assignm,ent to 
experimental and control groups ideally. Double- 
blind 


Randomization. Validity. Reliability. Sample size. 
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/\ugiet20O7 



Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Define 1 type research 
des 


Case studies: qualitative research that can provide 
an impressionistic and holistic understanding of 
problem, hut not a definitive/ Correlational: 
statistical correlation between phenomenon does 
not necessarily imply a causal relationship between 
them. True exp: under ideal conditions, allows a 
more confident determination of causality 


Corelational: relationship between variables, ex- 
post facto not causal. True experiments: 
randomized 




Diff exp & control group 


An exp group is randomly assigned to the 
particular condition or practice that is being tested. 
The control group does not. 


Exp receive the intervention; the other doesn’t/ 




Random assign import? 


To avoid self-selection. People who choose to be in 
a study are very likely different than the general 
population. 


Avoid self-selection that can create differences in 
groups being tested. 




Now use/like to use 
eviden 


4 


4 




How access research 


NCSALL, NIFL 


NCSAEE, NIFE, CAEEA 




?s ask about partic study 


Methodology. Relevance of research question. 


Methodology. Support by other studies. Quality of 
data. Peer reviewed. 




Key concept take away 




Importance of promoting research within 








programs. Innovation comes from within. 
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Define ebp 


Practices based on research data. 


Supporting data that supports a concept or idea 




2 features sbr 


Tested over time (reliable). Implementable (valid) 


Empirical. 




Define 1 type research 
des 


Case studies: based on actual situations 


Case studies: based on actual situations 




Diff exp & control group 


Exp; no common elements. Control: some 
elements are. 


Control group will be observed with specific 
common elements to evaluate results or change 




Random assign import? 


To provide a “fair sampling” of a large group 


Helps understand a larger group. 




Now use/like to use 
eviden 


4 


4 




How access research 


Through our Far West Great Center and other 
conferences 


Different websites. 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




?s ask about partic study 


What impacts student learning most? Other than 
purposeful and meaningful, what keeps adult 
learners engaged in teaming? 


Recognize the needs of the center or community. 


Key concept take away 




Alignment between research, instruction and 
assessment. 


66 


Define ebp 


Practice based on prior research 


Practice supported by prior research 


2 features sbr 


Experiments. Methodology. 


Examples. Percent of success rate. 


Define 1 type research 
des 


Case studies: research based on previous cases 
with its own set of methods 


True experiments: experiments that answer the 
question “How does it work?” 


Diff exp & control group 


Control group: one has set rules, methods, criteria, 
participants. Experimental: there is not an already 
established criteria. 


Control group has set of controls, whereas an exp 
group is more open. 


Random assign import? 


Provides different views, ideas, perceptions. Gives 
a much broader and realistic result. 


Given a better view of the perspective of all 
participants. 


Now use/like to use 
eviden 


3 


4 


How access research 


Through internet, administration, other instructors. 


Through websites, instructional coordination and 
newsletters. 


?s ask about partic study 


Success rate? Population used in research study? 


Type of research. Type of population. Success rate. 
Resources needed. 


Key concept take away 




Silent reading strategy, evaluation and assessment 


67 


Define ebp 


Practice that is based upon observed or anecdotally 
reported results of strategies. 


Practice based upon research findings providing 
evidence of success. 


2 features sbr 


Control group data is included. Statistics are 
garnered through accepted research-based method 


Published in peer journal. Rigorously data-based. 


Define 1 type research 
des 


In case studies, history and current practice are 
included and results are compared and controlled 
to other methods with comparable histories. 


True experiments: Control group, participant 
group — measure results of methods used. Random 
selection of both groups. 


Diff exp & control group 


Exp is included in the method or strategy being 
studied. Control is not included in the method. 


Exp: receive the method/item, etc. Control; does 
not receive the method/item 
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Resp. 


Question 


Pre-Workshop 


Post-Workshop 




Random assign import? 


It mirrors real-world situations. 


No personal bias or other generally occurring 
“skews”. 




Now use/like to use 
eviden 


6 


6 




How access research 


Internet! Library, co-workers, referral, periodicals 


Websites provided, trusted sources primarily 




?s ask about partic study 


Do our students have needs that can be met 
through these findings and are not currently being 
met? Do we have the capacity to make this 
change? 


Does this address a local need? Is it reasonable 
within this structure? 




Key concept take away 




Importance of moving from “by the seat of your 
pants” instruction 


68 


Define ebp 


Utilizing teaching techniques based on research 


Utilizing techniques based on research and 
professional wisdom 




2 features sbr 


Scientifically calculated. Research/experiment 


Left blank 




Define 1 type research 
des 


True experiments: set up experiment with control 
group and experimental and compare results 


True exp: random selected group with both exp and 
control groups. The group (n) must be large 
enough to have statistically true readings. 




Diff exp & control group 


Exp is given the “instruction/medicine” and control 
group hasn’t 


Exp: receive the modification. Control group: 
blind, no 




Random assign import? 


Take out bias; make it objective 


Take out, reduce bias and increase objectivity. 




Now use/like to use 
eviden 


2 


4 




How access research 


Texts, periodicals, sharing stories with other 
teachers 


TCALL and all resources on handout 




?s ask about partic study 


Can you teach an adult a language or to read if they 
never went to school and had limited 
communication throughout childhood? Does the 
brain atrophy? 


None at this time. 




Key concept take away 




You can use research to make change. 
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